Improved Nyquist Filter and Method 

FIELD OF THE INVENTION 

The present invention relates generally to filter circuits and more particularly to 
an improved Nyquist filter and method. 

BACKGROUND OF THE INVENTION 

The present invention relates to filters such as the kind that can be utilized in 
communications systems. Figure 1 illustrates a basic block diagram of a digital 
conrnixmication system 10 that utilizes pulse amphtude modulation. In this system, a 
pulse generator 12 receives clock pulses and binary input data. The output of pulse 
generator 12 will be a digital binary stream of pulses. 

The pulse stream fi-om pulse generator 12 is applied to the digital transmitting 
filter 14 that shapes the pulse for output to the digital-to-analog converter 15 and 
transmission over channel 16. Channel 16 may be a wired or wireless channel depending 
upon the application. The transmitted data is received at receiving filter 18. The output 
of filter 18 is applied to analog-to-digital converter 20. Analog-to-digital converter 20 
utilizes clock pulses that are generally recovered firom the transmitted data by clock 
recovery circuit 22. The output binary data firom analog-to-digital converter 20 is a 
rephca of the input binary stream that was provided to pulse generator 12. 

Major objectives of the design of the baseband PAM system are to choose the 
transmitting and receiving filters 14 and 18 to minimize the effects of noise, to eliminate 
or minimize inter-symbol interference (ISI) and to reduce stop band energy. Inter- 
symbol interference can be theoretically eliminated by properly shaping the pulses of the 
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transmitted signal. This pulse shaping can be accomplished by caxising the pulse to have 
a zero value at periodic intervals. 

Modem embodiments of pulse shaping filters use a pair of matched filters, one for 
transmit and one for receive. The convolution of the transmit filter with the receive filter 

5 forms the complete pulse shaping filter. Inter-symbol interference is avoided since the 
combined filter impulse response reaches unity at a single point and is zero periodically 
at every other information point (Nyquist sampling rate). The linear superposition of 
pulses representing a pulse train preserves bandwidth and information content. Linear 
supeiposition of band limited pulses remains band Umited and sampling the combined 

10 filter at the information rate recovers the information. 

Figure 3b shows an example of a Nyquist filter impulse response. Zeros occur at 
the information rate, except at one information bearing point. All Nyquist filters having 
the same stop band are equally bandwidth limited if the time response of the filters is 
allowed to go to infinity. Realizable filters, however, are truncated in time since it is not 

1 5 possible to have an infinitely long time fimction. Truncation error in the time domain 
causes the theoretical stop band achievable by all Nyquist filters to be violated, so that 
out of band energy exists in excess of the stop band fi-equency. 

The most bandwidth efficient filter is the "brick wall" filter illustrated in Figure 
3a by the box (a=0). The time response of this filter is shown in Figure 3b (a=0). While 

20 bandwidth efficiency is theoretically greatest for a brick wall filter as the time response 
^proaches infinity, truncation error causes poor performance for practical and realizable 
approximations to the brick wall filter. 
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One method of producing practical filters is to allow the stop band of Nyquist 
compliant filters to exceed the bandwidth of the ideal brick wall filter and smoothly 
transition to the stop band. A class of such filters is the raised cosine filters. In the 
frequency domain (Figure 3a), the raised cosine filter smoothly approaches the frequency 
5 stop band (except for the limiting brick wall filter case). The raised cosine filter is 

continuous at the stop band and the first derivative is continuous. The second derivative 
of a raised cosine filter, however, is not continuous at the stop band. 

In current embodiments of most systems, the raised cosine filter is used in its 
matched filter version. The transmit square root raised cosine filter, which determines 
10 the spectral bandwidth efficiency of the system, is discontinuous in the first derivative at 
the stop band. 

SUMMARY OF THE INVENTION 

The preferred embodiment of the present invention utilizes a pulse shaping filter 
15 that meets the Nyquist criteria. This filter also has the property of being continuous in 
the frequency domain up to and including the first derivative for the square root matched 
filter version. Some embodiments of the invention are in fact continuous in all 
derivatives for the square root version and these filters are closer to the ideal brick wall 
filter for the same stop band, 
20 Nyquist filters are produced from filters in the frequency domain with a fixed 

frequency cutoff. As is well known in the state of the art, a fixed cutoff frequency leads 
to an unrealizable filter of infinite duration in the time domain. To produce a realizable 
filter, the ideal filter is approximated by time delaying and truncating the infinite unpulse 
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response. Truncation, however, produces unintentional out of band energy. One goal 
that is achieved by some embodiments of the present invention is to minimize this 
unintentional out of band energy after the filter is truncated. 

Embodiments of the present invention provide filters that give a smaller signal 

5 ripple at the truncation length for the same theoretical stop band as raised cosine filters 
and therefore have better attenuation in the firequency domain. Accordingly, the 
preferred embodiment of the present invention has better truncation performance than the 
raised cosine that represents state of the art design for identical theoretical stop band. 
The present invention includes embodiments that have several advantages over 

1 0 prior art Nyquist filters such as the raised cosine filter. For example, the filter of the 

preferred embodiment of the present invention reduces the effects of truncation errors, by 
reducing the energy remaining in the terms beyond the truncation length. This 
attenuation leads to lower energy levels in the stop bands. For example, one embodiment 
filter of the present invention has been shown to provide lOdB improvement in the fiUer 

1 5 stop band, as compared with a comparable raised cosine filter. In other words, the out-of- 
band transmissions are reduced by 90%, a significant improvement. 

Implementation of embodiments of the present invention in a communication 
system provides enhanced system performance. Since the out-of-band performance is 
improved, adjacent channels can be moved closer together and use less firequency guard 

20 band. This feature leads to more efficient use of the available bandwidth. This 
advantage will be similarly be attained for subchannels within a channel. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The above features of the present invention will be more clearly understood from 
consideration of the following descriptions in connection with accompanying drawings in 
which: 



Figure 2 is a block diagram of a QAM communication system that can utilize the 
teachings of the present invention; 

Figures 3 a and 3b are the frequency and impulse response characteristics of a 
raised cosine filter; 



Figure 5 is a plot comparing the frequency response of a known raised cosine 
filter with a preferred embodiment Nyquist filter of the present invention; 

Figures 6a and 6b are an impulse response of a filter of the present invention; 
Figures 6c and 6d compare impulse response for two embodiment filters of the 
1 5 present invention; 

Figure 7 is a generalized diagram of an RF communications system that can 
utilize the present invention; 

Figures 8a-8c show an exemplary base station of the system of Figure 7; and 
Figures 9a-9b show an exemplary terminal unit of the system of Figure 7. 

20 

DETAILED DESCRIPTION 

The making and use of the presently preferred embodiments are discussed below in 
detail. However, it should be appreciated that flie present invention provides many 
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Figure 1 is a block diagram of a known digital communication system; 



10 



Figure 4 is a Nyquist filter shown as being represented by three regions; 
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applicable inventive concepts that can be embodied in a wide variety of specific contexts. 
The specific embodiments discussed are merely illustrative of specific ways to make and use 
the invention, and do not limit the scope of the invention. 

The present invention will first be described with respect to a quadrature amplitude 
modulation (QAM) system. An improved class of filters will be described with respect to 
this system. A number of other applications that could also utiUze a filter of the present 
invention are then discussed. 

Figure 2 illustrates a block diagram of a QAM system 100 that can utihze the filter 
of the present invention. As shown by blocks 102 and 104, the system can be used for either 
analog or digital data, or both. If an analog signal, such as voice and/or video as examples, 
is to be transmitted, it will first be filtered with low pass filter (LPF) 106 and then converted 
to a digital signal by analog-to-digital converter (ADC) 108. The digital stream, whether 
fi-om digital source 104, ADC 108, or both, will then be mapped into I (in-phase) and Q 
(quadrature-phase) carriers in mapping unit 110. Each of these steps is well known in the 
art and will not be described in detail herein. For more information regarding these blocks, 
reference can be made to W. T. Webb and L. Hanzo, Modem Quadrature Amp litude 
Modulation, EEEE Press, 1994, chapter 3, pp. 80-93, incorporated herein by reference. 
Figures 2 and 3 were adapted fi-om this text. 

The I and Q streams will be filtered in Nyquist filters 1 12 and 1 12*. As will be 
discussed below, the present invention provides a Nyquist filter 1 12 that has enhanced 
performance compared to prior art filters. In general, a Nyquist filter has an impulse 
response with equidistant zero-crossings at symbol points. As a result, the filter eliminates 
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inter-symbol interference (ISI). More detail of the improved Nyquist filter 112 (112*) will 
be provided below. 

Once the I and Q signals have been generated and filtered, they are modulated by an 
I-Q modulator 1 14. The modulator 114 includes two mixers 116 and 118. As shown, mixer 

5 1 16 is used for the I channel and mixer 1 18 is used for the Q channel. The modulator 114 
causes both the I and Q channels to be mixed with an intermediate fi-equency (IF) signal, 
generated from signal source 120. The I channel will be mixed with a IF signal that is in 
phase with respect to the carrier and the Q channel will be mixed with an IF signal that is 90 
degrees out of phase. This process allows both signals to be transmitted over a single 

10 channel within the same bandwidth using quadrature carriers. 

The analog signal output from the modulator 1 14 is then frequency shifted to the 
carrier frequency by modulator 124. In the preferred embodiment, the carrier frequency is 
in the radio frequency (RF) range, but other frequencies coxild be used. The present 
invention would equally apply to any system that uses CDMA (code division multiple 

1 5 access), TDMA (time division multiple access), optical systems, HDTV (high definition 
television), cable systems and others. 

Returning to Figure 2, the RF signal is transmitted to the receiver through a channel 
126. This channel can be wireless, e.g., RF wireless communications. Alternatively, the 
channel could be an electrical connection or an optical connection. 

20 A demodulator 128 at the receiver demodulates the received signal by mixing it 

down to the IF for the I-Q demodulator 130. The I-Q demodulation takes place in the 
reverse order to the modulation process. The signal is spUt into two paths, with each path 
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being mixed down with intermediate frequencies that are 90° apart. The two paths are then 
provided to Nyquist filters 132 and 132*, which can be of the type described below. 

The output of the Nyquist filters 132 and 132* is provided to a demapping unit 134 
which returns the signal to a digital stream. If the original data was digital, then the data 

5 from digital source 104 should be recovered at digital output 136. If the original signal was 
analog, on the other hand, the digital stream from demapping unit 134 would be retumed to 
analog form by digital-to-analog converter (DAC) 138. This analog output of DAC 138 
could then be filtered by low pass filter 140 and provided to analog output 142. Once again, 
reference can be made to the Webb and Hanzo text for additional details on QAM systems. 

1 0 As mentioned above, a Nyquist filter has an impulse response with equidistant zero- 

crossings at sampling points to eliminate inter-symbol interference (ISI). Figures 3a and 3b 
show the frequency characteristic (Figure 3 a) and impulse response (Figure 3b) of a weU 
known Nyquist filter, the raised cosine filter. Nyquist showed that any odd-symmetric 
frequency domain extension characteristic about f^ and (-fw) yields an impulse response 

1 5 with a unity value at the correct signaling instant and zero crossings at all other sampling 
instants. The raised cosine characteristic meets these criteria by fitting a quarter period of a 
frequency-domain cosine shaped curve to an ideal (brick wall) filter characteristic. 

The parameter controlling the bandwidth of the raised cosine Nyquist filter is the 
roll-off factor a. The roll-off factor a is one (a==l) if the ideal low pass filter bandwidth is 

20 doubled, that is the stop band goes to zero at twice the bandwidth (2fN) of an ideal brick wall 
filter at fw. If a=0.5 a total bandwidth of 1 .5fN would result, and so on. The lower the value 
of the roU-ofif factor a, the more compact the spectrum becomes but the longer tiine it takes 
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for the impulse response to decay to zero. Figures 3a and 3b illustrate three cases, namely 
when a=0, a=0.5 and a=l .0. 

The equation defining the raised cosine filter in the firequency domain 
(NFRaisedCosine(f) where f is fi-equency) and the corresponding impulse response 
5 (nfRaiscd cosinc(t) whcte t is time) are defined by the following equations. 
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Matched filters are used in many communication systems in order to maximize the 
signal-to-noise ratio. As illustrated in Figure 2, the matched filtering can be accomplished 

10 by including Nyquist filters at both the transmitter (filter 1 12) and the receiver (filter 1 32). 
Since two filters are provided, each will have a characteristic of square root of a Nyquist 
fimction in the firequency domain. In this manner when the effect of both filters is taken into 
consideration, the desired Nyquist characteristic will be achieved. In other words, the 
product in the fi-equency domain of the two matched filters is equivalent to the fi'equency 

1 5 domain representation of the Nyquist filter. The following equations provide the firequency 
domain and impulse response of the sqixare root version of the known raised cosine filter. 
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Spectral efficiency can be gained using filters that are Nyquist compliant and that 
smoothly transition to the stop band in the jfrequency domain. Figure 4 shows three 
approximate regions of a Nyquist filter in the frequency domain. Region I constitutes the 
unattenuated passband. Region 11 represents the transition band and Region III represents 
the stop band. Filter smoothness can be measured by the number of derivatives with 
respect to frequency that remain continuous. In particular, for a good filter performance, 
the filter should be smooth at the point between the transition and stop band regions 
(n and n) and at the point between the pass band and transition regions (I and 11). The 
square root raised cosine is discontinuous in the first derivative at the point between 
regions n and EI. 

The envelop of the time domain response decays more rapidly if the frequency 
response is smooth, that is continuously dififerentiable. Unfortunately, the frequency 
response of a raised cosine filter is non-ideal when the square root is taken. In particular, 
when the square root of a raised cosine filter is taken, the first derivative is discontinuous at 
the boundary of regions n and IE. As a result, the frequency domain curve does not come in 
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smooth at the stopband as in the raised cosine case (see Figure 3a). Instead, the curve comes 
in sharply to the stopband. A discontinuity in the frequency domain can lead to higher 
peaking at longer duration in the time domain. 

Figure 4 can be used to illustrate this concept. As discussed above, a Nyquist filter 
5 (either of the prior art or of the present invention) is shown as being divided into three 
regions. Region I is the pass band, region n is the transition band and region EI is the stop 
band. The transition points between regions, labeled 405 and 410, are the points of greatest 
concern since the functional form of the frequency domain response is defined by different 
equations at these points. One goal of the preferred embodiment of the present invention is 

10 that the filter fimction be not only continuous but also smooth (i.e., continuous for at least 
the first derivative) in both the Nyquist filter and square root Nyquist version. 

The present invention provides a class of Nyquist filters that have better stop band 
performance than the known raised cosine filter. The Nyquist filter of the preferred 
embodiment of the present invention comes in smooth at the stopband when the square root 

15 version is used. In other words, the first derivative of this fimction is continuous. This 
property allows for truncation at some delay with less energy loss than the known raised 
cosine. 

In the preferred embodiment, an impro ved class of Nyquist filters will meet two 
criteria. First, the filter will meet the Nyquist criteria, that is the frequency domain will have 
20 odd symmetry about the cutoff frequency (fw). Second, the square root version of the 

frequency domain of the filter will be continuous at least in the first derivative, preferably at 
all points. In a subclass of filters of the present invention, all derivatives will be continuous 
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at all points. In typical applications, the higher order derivatives will all be equal to zero at 
the transition point between regions n and HI and between I and H. 

A number of filters meet the criteria for the class of filters of the preferred 
embodiment of the invention. Examples include all Nyquist compliant functions that when 
5 differentiated can be written in the fomi 
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with stop band 7r/T(l-i-a). Here derivatives to all orders are smooth at the stop band giving 
zero (using the product rule for differentiation). As will be discussed in further detail below, 
preferred examples of such functions are composite sine and cosine functions. Other 
1 0 functional fomis may also meet this criteria. 

The first example of an improved Nyquist filter has a firequency domain defined by a 
composite sine function. In particular, the frequency characteristic of this preferred filter is 
provided by the following equations: 
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The square root of the firequency domain is given by the following equations. 
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This equation was derived by starting with the raised cosine frequency domain 
function and taking nil times the sine term of that raised cosine function as the argument of 
a second sine function. Accordingly, the frequency characteristic is a composite function 
since it includes a sine of a sine. This modification was made because when the square root 

5 version is differentiated, the frequency domain function is smooth at the stop bands. This 
function is continuous in the frequency domain up to and including the first derivative since 
the function and the first derivative of the function are continuous. 

The time response can be derived by inverting the frequency function. This is done 
by taking the theoretical Fourier transform of the frequency function and leads to the 

1 0 following response: 
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The impulse response of the square root Nyquist filter can be expressed as 



k4t \t) \t) 
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15 It is noted that mathematically inverting the frequency domain function leads to an 

infinite number of terms. This is not a problem, however, since the higher orda: terms 
decrease rapidly. In effect, the wavefomi can be calculated using only the first few terms. 
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As can be seen from the above equation, the impulse response includes Bessel 
functions J(x). In a digital computing system, these functions can be derived in the same 
manner as any other function, such as a sine. As a result, the present invention is no more 
difficult to implement than any other filter. 
5 In the preferred embodiment, the Nyquist filter is implemented as a digital filter. 

Accordingly, the impulse response equation can be calculated once and the results stored in 
a lookup table. As a result, the fact that the equation is computationally complex is not a 
detriment. Since the equation is solved off-Une, there is little negative impact if the 
computation is time consuming. Of course, this fact does not prevent the impulse response 
10 fi-om being calculated in real-time if a system was so designed. 

The performance characteristic of the first embodiment filter is illustrated in Figure 5 
along with curves from a square root raised cosine filter where a = 0.17 and a=0.2. Each of 
these curves are taken in the square root version. These curves were taken from simulations 

of a system as described in co-pending apphcation Serial No. (COM-002). In 

1 5 that system, a roll-off factor a=0. 17 was found to be optimal for a raised cosine filter. As 
demonstrated by Figure 5, the performance of the new Nyquist filter is improved at the 
stopbands, i.e., outside the ideal bandwidth of the channel. As shown in the figure, the filter 
of the present invention has lOdB lower transmissions at the stop band when truncated to 
eight symbol periods. This represents a significant improvement. 
20 Improved stop band performance is beneficial in a commimication system since it 

allows signals on adjacent frequency channels to be more compactly fit into the frequency 
spectrum. It also makes it easier to meet the emissions mask requirements for a given 
channel. These mask requirements are generally dictated by a regulatory authority, such as 
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the Federal Communications Conmiission in the United States, and define the level of signal 
that can be allowed outside a given channel. 

In addition this filter eventually has lower amplitude ringing in the tails than a raised 
cosine filter. The new class of filters are characterized in the time domain by the tails 
5 damping faster than the equivalent raised cosine filter tails. After a given time delay in the 
impulse response, the new Nyquist filters ring at a lower amplitude and continue to ring at a 
lower amplitude than the equivalent raised cosine filter at all longer times to infinity. 

Figures 6a and 6b illustrate an exemplary impulse response for the new Nyquist 
filter (curve 610) and a known raised cosine filter. Referring to Figure 6b in particular, the 
1 0 new Nyquist filter exhibits much lower amplitude levels at times farther firom the peak. 

Because of these lower amplitudes, less energy is lost by truncation and therefore truncation 
will have a smaller affect on the firequency domain (and thus the stop band improvement 
shown in Figure 5). In the filter of the preferred embodiment, the time domain is truncated 
after eight symbol-time delays, e.g., after 8T where T is the symbol rate. 
1 5 A second embodiment filter of the present invention will now be described. This 

filter exhibits each of the characteristics of the first embodiment filter. That is, the filter 
meets the Nyquist criteria and, in the square root version, is continuous for all values of the 
first derivative. In addition, this filter is also continuous for all values of higher order 
derivatives as well. 

20 The firequency domain and square root firequency domain equations for the second 

embodiment filter can be expressed with the following equations: 
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Inversions of the square root filters for the first and second embodiments are shown 
in Figure 6c (and Figure 6d, which focuses on the tails). The second embodiment inversion 

5 was performed numerically. Both filters have the same theoretical stop band. The second 
embodiment exhibits the same behavior with respect to the first embodiment as the first 
embodiment does with respect to the raised cosine filter. As shown in the figure, the first 
embodiment rings less at the beginning than the second embodiment. As shown in Figure 
6d, however, the second embodiment damps out more quickly giving less energy in the tails. 

1 0 The. second embodiment transitions more smoothly to the stop band than the first 
embodiment. 

Two examples of filters that meet the criteria for tlie new class of Nyquist filters 
have been described. Other fimctional forms may also meet the criteria of continuous 
derivatives and can be considered for this type of filters. These expressions might include 
15 hyperbolic sines and cosines, polynomials, and ecliptic fimctions. 
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Alternatives to the prefeixed embodiments also include linear combinations of temis 
in the transition band. For example, a iBlter could be derived by adding the frequency 
domain response of the first embodiment described above with the frequency domain 
response of the second embodiment described above. These two (pr more) functions could 
be weighted evenly or not. The temis could each include a differing number of sine terms in 
the combinations. Filters with transitional region (region n in Figure 4) terais like 
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and square root versions of the same, where a is to be determined, and linear combinations 
of these terms, can be fashioned to meet Nyquist's criteria. 

1 0 Filters as described herein can be used in a variety of applications. For example, the 

filters can be used in any system that utiUzes pulse shaping filters. Digital communication 
systems provide one such example. For example, filters of the present invention could be 
used in wireless communications (cellular, GSM, microwave, satellite), wired 
communications (in telephone systems, cable modems), optical systems, broadcast systems 

1 5 (digital television/radio, satellite), and others. 

One example of a system that can utilize a filter of the present invention is described 

in co-pending application Serial No. (COM-002), which is incorporated herein 

by reference as if reproduced in its entirety. A Nyquist filter of the present invention can be 
used in place of each Nyquist filter included in that system. This system will now be 

20 described with respect to Figures 7-9. 

Figure 7 illustrates an exemplary radio system 700. System 700 could be a 
cellular telephone system, a two-way radio dispatch system, a localized wireless 
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telephone or radio system or the like. Base unit 702 can conununicate over transmission 
medium 704 to one or more terminal xmits 706. Transmission medium 704 represents the 
wireless communication spectrum. Terminal units 706 can be mobile units, portable 
units, or fixed location units and can be one way or two way devices. Although only one 

5 base unit is illustrated the radio system 700 may have two or more base units, as well as 
interconnections to other communication systems, such as the public switched telephone 
network, an internet, and the like. In the preferred embodiment, the system provides for 
full duplex communications. The teachings of the present invention, however, apply 
equally to half duplex systems, simplex systems as well as to time division duplex and 

10 other two-way radio systems. 

Details of preferred embodiments of the base unit and terminal units are provided 
in the following descriptions. Figures 8a through 8c provide block diagrams of several 
major components of an exemplary base unit 702. A skilled practitioner will note that 
several components of a typical radio transmitter / receiver not necessary to an 

15 understanding of the invention have been omitted. Note that many of the features and 
functions discussed below can be implemented in software running on a digital signal 
processor or microprocessor, or preferably a combination of the two. 

Figure 8a illustrates the four sub-chaimel architecture of a base unit 102 operating 
in transmitter mode. The following explanation will be addressed to sub-channel A, 

20 although the teachings apply to the other sub-channels as well. Sub-channel A includes 
"red" signal coding block 802 and '*blue" signal coding block 804. The "red" and "blue" 
are arbitrary designations for the first and second time slots. Details regarding the signal 
coding blocks are provided in the co-pending apphcation. For the present purpose, it is 
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sufficient to state that the signal coding blocks receive voice and/or data signals, encode 
those signals if necessary, combine control signals, and prepare the combined signals for 
passage to QAM modulator 806. 

Modulator 806 modulates the received signal using a quadrature amplitude 

5 modulation (QAM) architecture employing a 16 point constellation. With a sixteen point 
constellation, each symbol mapped to the constellation represents four bits. In the 
preferred embodiments, the signal is differentially encoded using a differential Gray code 
algorithm. Details of such architectures are well known in the art. See, for instance, 
Webb et al., Modern Quadrature Amplitude Modulation (IEEE Press 1994). Various 

10 other QAM techniques are known in the art, including Star QAM, Square QAM, and 

Coherent QAM. Additionally, other encoding techniques could be used, such as Okunev 
encoding or Khvorostenko encoding could be used in Ueu of Gray coding. Other 
embodiments of the invention could include other modulation techniques as are known in 
the art, provided the modulation provides for sufficient data rate (16 kb/s in the preferred 

15 embodiments) with acceptable signal quality (i.e. signal to noise ratio) for the desired 
application. 

The in-phase and quadratxire-phase components of the QAM modulated signal 
(illustrated by the single line representing both signals, as indicated by the slash through 
each such signal path) are then passed to Nyquist filter 808 which provides a pulse 
20 shaping filter in order to limit the overall bandwidth of the transmitted signal. In the 
preferred embodiment, the Nyquist filter operates at a 65 times over-sampling rate, in 
order to simpUfy the analog filtering of digital images. Nyquist filter 808 may comprise 
any of the filters discussed in this specification. 
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Additionally, the signal is multiplied in sub-channel offset block 810 by the offset 
required for the sub-channel upon which the signal is to be transmitted. In the preferred 
embodiment, the offset for sub-channel A would be minus 7.2 kHz, for sub-channel B 
would be minus 2.4 kHz, for sub-channel C would be plus 2.4 kHz, and for sub-channel 

5 D would be plus 7.2 kHz. 

The QAM modulated and filtered signal A is then combined with the modulated 
and filtered signals firom sub-channels B, C, and D in sub-channel summer 812 before 
being passed to digital-to-analog converter 814 where the combined signals are converted 
to an analog signal. The signal is then passed to radio fi-equency circuitry (not shown) 

10 where the signal is modulated to RF and ampUfied for transmission, as is known in the 
art. 

In the preferred embodiment, the system is implemented using a digital signal 
processor. In tiiis embodiment, all of the circuitry in the box labeled "Subchannel A" (as 
well as the other subchaimels) and the subchannel summer 812 are implemented by a single 

15 chip. In fact, this chip can be designed to handle analog signals at both the input and the 
output by integrating an analog-to-digital converter (not shown) and digital-to-analog 
converter 8 14 on chip. Altematively, the fimctions can be distributed amongst a number of 
interconnected integrated circuit chips. 

In the preferred embodiment, the impulse response of filter 808 would be calculated 

20 once and stored in a lookup table, for example a EEPROM on a digital signal processor 
chip. The filter can then be implemented in the same manner as any other finite impulse 
response (FIR) filter, as is known in the art. 
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Figures 8b and 8c illustrate base unit 102 operating in receiver mode. Figure 8b 
provides a high level block diagram of the four sub-channel architecture. Signals from 
the terminal units are received by radio frequency (RF) receiving circuitry (not shown). 
A/D converter 840 receives the signal from the RF receiving circuitry and converts it to a 

5 digital signal, which is fed to each of the four sub-channel paths, 844, 845, 846, and 847. 
Figure 8c illustrates the details of sub-channel A, 844 of Figure 8b. Note that 
these teachings apply equally to sub-channels B, C, and D, 845, 846, and 847, 
respectively. In complex multiplier 850, the frequency offset corresponding to the 
particular sub-channel (+/- 2.4kHz or +/- 7.2kHz) is removed from the incoming signal. 

10 The signal is then frequency channelized by the square root Nyquist matched filter 852. 
Once again, matched filter 852 may be any one of the filters discussed herein. 

The filtered signal is passed to Symbol Synchronization block 854, which 
calculates the proper sampling point where there exists no (or minimal) inter-symbol 
interference signal. This is accomplished by calculating the magnitude of the sample 

15 points over time and selecting the highest energy points (corresponding to the 

synchronized symbol sample points). Magnitude tracking (block 858) is performed in 
order to remove channel effects from the differential decoder by determining if a detected 
change in amplitude of the signal is based on the intended signal information or on fading 
of the signal caused by interference. Based upon this determination, the threshold by 

20 which an incoming pulse is considered to be on the outer or inner ring of the QAM 
constellation (logically a "1" or a "0") is modified to adapt to the changing incoming 
signal quality. 
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Based upon the information provided by magnitude tracker 858, the likelihood 
that a bit is in error is calculated in fade finder block 859. Blocks determined to be a high 
risk of being in error are marked as "at-risk" bits in block 861 . The "at-risk" bit 
information is fed forward to the appropriate one of red or blue signal decoding blocks 

5 866 and 869 and is used by the decoding blocks' error correcting processes. Symbol 
synch block 854 also feeds phase tracker 860, which is discussed below. 

The modulated signal is fed firom magnitude tracker 858 to QAM modulator / 
demodulator 864, wherein the signal is de-modulated to the base band signal before being 
passed to the appropriate one of red or blue signal decoding paths. Note that only one 

10 functional block is shown in Fig. 8c for each element of the path from Nyquist filter 852 
to the red / blue decoding blocks. In practice, however two dupUcate paths exist, one 
each for the red and blue signals. As a consequence Red^lue multiplexer 863 is 
provided in the feedback path between Automatic Frequency Control block 862 and 
complex multiplier 850. This is because a different firequency correction factor will be 

15 determined for the incoming red and blue signals. The appropriate correction factor must 
be fed back to the complex multipUer when the desired signal (red or blue) is being 
received. Frequency control is provided in phase tracker 860 and AFC block 862, 

Details of the terminal unit are provided in Figures 9a and 9b. As noted above, 
the following discussion provides additional details and is relevant to the description of 

20 the base unit as both units use similar schemes for voice coding, signal processing, and 
modulation. Figure 9a illustrates in block diagram form the terminal unit acting as a 
transmitter. The end users audio input is received at microphone 902 and passed to 
vocoder 904 via codec 903. Vocoder 904 provides for coding, compression, and forward 
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error correction functions, as discussed above with reference to Figure 8b. The signal is 
then passed to TDM formatter 908 along with control and synchronization bits from 
block 906 as well. The combined signals from vocoder 904 and control and synch block 
906 are up-converted in TDM formatter block 908 to double the data rate. The signal is 
5 then passed to channel coder 910, where control and sync information is added to the 
signal and the bits are interleaved in order to make the transmitted signal less susceptible 
to noise, as is well known in the art. 

The signal is modulated using QAM modulation(as described above) in block 
912, as described above with reference to Figs. 8a through 8c 

10 The in-phase and quadrature-phase components of the QAM modulated signal are 

then passed to Nyquist filter 914 which provides a pulse shaping filter in order to limit 
the overall bandwidth of the transmitted signal In the preferred embodiment terminal 
unit, the Nyquist filter operates at a 65 times over-sampling rate, in order to simplify the 
analog filtering of the digital image. The Nyquist filter 914 can be any of the filters 

1 5 discussed herein and will be matched with the corresponding filter from the base unit. 

After passing through the Nyquist filter 914, the signal is multiplied by the 
frequency offset required for the sub-channel upon which the signal is to be transmitted. 
The offset signal is suppUed by sub-channel offset block 926, which selects the sub- 
channel offset based upon instructions received from the base unit or upon pre- 

20 programmed instructions contained within the terminal unit's non-volatile memory. 

Note that only a single QAM modulator is required for the terminal unit. This is 
because the terminal unit will only transmit on one sub-channel at any give time, as 
opposed to the base unit, which broadcasts over all sub-channels simultaneously. 
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The in-phase signal is fed to D/A converter 916 and the quadrature component is 
fed to D/A converter 918 where the signals are converted to analog signals. Filters 920 
and 922 filter out spectral images at the Nyquist rate. Finally, the signals are fed to I & Q 
modulator 924 where the signals are modulated to radio frequency before being passed to 
RF transmitting circuitry (not shown). 

Figure 9b illustrates the terminal unit functioning as a receiver. Signals from the 
base unit or another terminal unit are received by RF receiving circuitry 930 where the 
RF signal is down-converted and filtered before being passed to A/D converter and mixer 
933 for the in-phase component and 934 for the quadrature-phase component. Also at 
this point, the frequency offset associated with sub-channel selection is removed from the 
signal components by mixing into the received signal a signal complementary to the 
offset signal. The complementary offset signal is determined by sub-channel frequency 
offset control information, as illustrated by block 926 and depends on the sub-channel 
upon which the terminal unit is receiving. The digital signals are passed to a matched 
square root Nyquist Filter (which comprises any of the embodiments described herein) 
and then demodulated to a real binary signal in demodulator 932. The digital binary 
signal is then de-coded in blocks 944 and 946 using Viterbi decoding. The signal is then 
de-multiplexed in the time domain, wherein the data rate is reduced from 16 kb/s to 8kb/s 
before being converted to an analog audio signal and reproduced by a speaker or similar 
end-user interface (not shown), or in the case of data before being displayed on an end- 
user interface such as an LCD display. Note that only one time slot 950 or 952 will be 
active at any given time and will drive the end-user interface(s). 
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Continuous fine firequency control is also provided for as indicated by blocks 860 
and 862 of Fig. 8c and block 940 of Fig. 9b. Slot and symbol synchronization is provided 
for in block 938. 

While this invention has been described with reference to illustrative embodiments, 
this description is not intended to be construed in a limiting sense. Various modifications 
and combinations of the illustrative embodiments, as well as other embodiments of the 
invention, will be apparrat to persons skilled in the art upon reference to the description. It 
is therefore intended that the appended claims encompass any such modifications or 
embodiments. 
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